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Bi-Ber Bilderkennungssysteme Berlin 
Muggelseedamm 221, 12587 Berlin 

Method and apparatus for automated detection and checking of geometrical 
and/or textural features of an object 



The invention concerns a method as set forth in the classifying 
portion of claim 1 and an apparatus for carrying out the method. 

The invention lies in the technical field of optically checking various 
views of an object, which is to be carried into effect for example in the 

15 process of circuit production and assembly. 

In regard to circuit production and assembly it is particularly 
necessary to check the prints on the top side of circuits in terms of their 
existence there and their quality (discernibility) in order to prevent 
confusions from occurring. In addition it is necessary to measure the 

20 connections (terminal or connection legs') which are disposed at the 
underside of the circuits, to ascertain whether they all lie within a 
predetermined tolerance range in one plane so that for example in a 
subsequent soldering process they can be simultaneously and reliably 
involved with the solder. That checking operation or ascertaining the 

25 deviation of the ends of the terminal legs from a flat contact surface is 
referred to as coplanarity checking. This coplanarity checking operation can 
be effected from a side view on to the rows of terminal legs at the sides of 
the circuit. Besides the above-mentioned print quality assessment and the 
coplanarity checking, the housing bottom clearance of a component and 

30 pitch, twist or skew, width, position, length deviation and contact point of 
the terminal connections are measured in the context of a so-called Mead 
and mark inspection'. 
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Automatic reading of plain text on products is state of the art; see 
for example R Koy-Oberthur: 'Ubersicht industrielier Anwendungen der 
Klarschrift- und Barcode-identifikation', Symposium: Aktuelle 
Entwicklungen und Realisierungen der Bildverarbietung, 11th and 12th 
5 September 1997, Aachen, Ministerium fur Wirtschaft, Mittelstand und 
Technologie des Landes Nordrhein-Westfaien. The aim pursued by that 
procedure is to recognise the individual symbols contained in the script, but 
not to derive an assessing measurement in terms of the readability 
afforded in any way thereby. It is precisely this however that is necessary 

10 and appropriate for such situations if the content of the printing is known 
per se but its potential readability has to be guaranteed, for example for 
liability and monitoring situations. 

The lecture by T Schroeter: 'Einsatz der Bildverarbeitung zur 
Druckvollstandigkeitskontrolle 1 at the same Symposium described a method 

15 which detects the completeness of printing, that is to say checks it for 
missing characters. That method makes use of the so-called histogram 
information. The method described herein assesses the printed area 
without deriving an evaluation in terms of the readability of the 
identification. 

20 It is also known to implement a coplanarity check on circuits by 

means of optical methods with which gap widths can be measured. In the 
procedure described by Qtec in 'Industrielle Bildverarbeitung/Maschine 
Vision', VDM, Robotik Automation, Maschinenbau Verlag GmbH, 
Frankfurt/Main. 1997, an image is taken from the underside of the circuit to 

25 be checked, that image permitting a view on to shadows cast by the rows 
of connection legs. A disadvantage in this respect is that evaluation is 
effected only from one image which is recorded directly from an individual 
side of the object to be monitored, and thus it is not possible also to 
implement a print quality check. The location of the shadow of a connection 

30 leg to be measured, on a base surface, is moreover influenced not only by 
the distance thereof from the base surface (on which the shadow is cast) 
but also the location of the connection leg over that base surface, and 
moreover the specific shape of the connection leg, from which major errors 



can result. Furthermore, with this procedure, a plurality of individual light 
sources are required to produce an evaluatable shadow image. 

The coplanarity monitoring unit SMD9000 from Trigon Adcotech uses 
five CCD-cameras for recording the images to be evaluated, for monitoring 
5 coplanarity, one for each side view and one for a plan view. A disadvantage 
in this respect is the high level of technical complication and expenditure 
arising out of the number of cameras required, and the fact that the 
individual views are not linked to each other. 

In addition WO 98/04882 discloses a method of the general kind set 

10 forth for the automated detection and checking of geometrical and textural 
features of an IC component in side views and a plan view. In that case 
detection of the features is effected using a CCD-camera and a 
corresponding storage unit and evaluation unit for image processing and 
image evaluation. In that case, quality and state assessment of the object 

15 is effected by a comparison with parameters which are predetermined in 
respect of the individual features, wherein the partial images of the object 
are substantially simultaneously recorded by means of a single CCD- 
camera by virtue of a suitable beam-deflection device and are at least 
partially assembled optically at the same time to form an overall image 

20 which shows all views and in which the boundaries of the partial images can 
be recognised. The overall image is then separately evaluated for checking 
the individual features in the boundaries of the partial images. In this case 
the coplanarity of the legs of the IC-component is also checked. That 
publication however does not disclose the form in which that evaluation 

25 procedure is implemented. 

Therefore the object of the present invention is to provide a method 
and an apparatus which make it possible at relatively low cost to detect and 
check in interrelated manner and reliably in relation to coplanarity 
structural and/or textural features of an object in a plurality of views, more 

30 especially in a plan view and in side views. 

In regard to its method aspect that object is attained by a method 
having the features of claim 1 and in regard to its apparatus aspect by an 
apparatus having the features recited in claim 7. 



3 



AMENDED PAGE 



The invention involves the technical teaching of obtaining from a 
single viewing position by means of suitable optical means an overall image 
constructed from partial images of various views of the object with a level 
of resolution which is sufficient for simultaneous structure and texture 
checking. 

The partial images are substantially simultaneously recorded and at 
least partially brought together optically at the same time to form an 
overall image showing all views, in which the boundaries of the partial 
images can be seen, and that is evaluated in the boundaries of the partial 
images separately - that is to say with different evaluation means or in 
terms of different parameters. 

In accordance with the invention firstly in the regions of the overall 
image which show side views on to the object, analysis of the gray value 
distributions is effected to ascertain locations in which parts (to be checked 
in terms of coplanarity) of the object come very close to a substrate. 
Subsequently the light quantity which passes through between the object 
and the substrate and which is reflected at the pixels as an intensity value 
is detected and, utilising the intensity values, the local light quantity 
pattern or configuration which characterises the width of a gap between the 
object and the substrate is determined. Finally the light quantity pattern or 
configuration can be converted into a gap width in accordance with a 
predetermined algorithm, utilising calibration information. The rule for 
conversion of an intensity pattern as between object and support is 
advantageously implemented on the basis of a spline approximation, the 
determination of rise values and calibration, in such a way that an area 
proportion under the spline curve determines the value of the gap width 
between the object and the support surface. That procedure is expedient 
especially for coplanarity checking of circuits or similar objects. 

In a preferred embodiment all partial images are assembled optically 
and recorded by precisely one image-recording device, wherein in the 
overall image the regions of the partial images are so positioned and 
characterised, in particular using the storage unit and/or the evaluation 
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unit, in such a way that they can be associated for the evaluation 
procedure with the individual views. 

Checking the object is made easier in many uses if in at least one 
additional step the scene is recorded without an object and/or with a 
5 reference object which has predetermined parameters in regard to the 
features, and the corresponding overall image is stored in the storage 
device for comparison and calibration purposes. 

In a region of the overall image (which for example shows the plan 
view of a circuit), for the purposes of detecting a texture and especially for 

10 checking the readability of a characterisation or identification, preferably by 
means of image processing, using convolution filters, areas with severe 
local intensity differences are emphasised, detected and quantified in terms 
of their dimensions, the result of the quantification operation in the above- 
mentioned areas is compared to given values and subsequently quality 

15 information is derived in respect of the texture or identification. 

In the apparatus according to the invention there is provided a flat 
support surface for the object, and the beam-deflection means are 
substantially arranged in the plane of the support surface in such a way 
that there is a view parallel to the support surface, permitting checking of 

20 the coplanarity of a plurality of parts of the object, which are towards the 
support surface. 

The apparatus according to the invention preferably includes a single 
image-recording device, relative to which the object is positioned in such a 
way that it fills only a part of its field of view, wherein arranged in 

25 remaining parts of the field of view are beam-deflection devices which 
produce images of side views of the object on the image-recording device. 

The beam-deflection devices are so positioned that they respectively 
produce an image of a given view of the object, but do not influence the 
other views (especially the plan view on to the object). 

30 The means for beam deflection have in particular prisms or mirrors 

which are fixed or displaceable and which can have surfaces which are 
optionally curved for specific uses. In addition or alternatively thereto they 
may have a light guide arrangement (one or more glass fiber bundles). 
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Associated with at least one of the beam-deflection devices are 
means for altering the imaging scale of at least one partial image with 
respect to at least one other partial image, in particular a lens 
arrangement. 

Furthermore the apparatus preferably includes - especially for 
coplanarity checking procedures - a lighting device which in particular has a 
light diffuser device for producing a regular light flux under the object, 
which is arranged behind (from the point of view of the beam-deflection 
device) projecting parts of the object. 

In that case the light diffuser device can preferably be interrupted in 
such a way that the free spaces permit a view on to the underside of the 
object. 

For recording a plan view, in a preferred feature a further lighting 
device which is suitable specifically for evaluating the recognisability of a 
surface texture is then arranged over the object. 

A preferred apparatus is one in which the image-recording device 
and the storage and evaluation unit are integrated in a structural unit. That 
can be for example a so-called 'smart camera*, as however is hitherto not 
used for lead and mark inspection. The advantages of using an integrated 
component are that on the one hand it is easily obtainable and on the other 
hand it can be easily configured for use in the context of the procedures 
described hereinbefore. 

Advantageous embodiments of the invention are also set forth in the 
appendant claims, and the description hereinafter with reference to the 
accompanying drawings in which: 

Figure 1 shows a diagrammatic simplified view of the overall 
structure of an embodiment of an apparatus according to the invention, 

Figure 2 shows an overall image of an example of an object to be 
checked, with a plan view and side views recorded from a viewing position, 

Figure 3 is a view of part of a further embodiment with 
advantageously arranged lighting devices for producing top light and 
relative transillumination, 
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Figure 4 shows a view of the intensity profile of a gap section 
between a support and a projecting part of an object, which is towards 
same, 

Figure 5 shows a side view and a view from below of a component 
5 with L-shaped connections, and a side view and a view from below of a 
component with J-shaped connections. 

Figure 1 is a diagrammatic view showing the principle of the overall 
structure of an apparatus for checking features, which can be recognised in 
side views and a plan view, of an object 1, from a single viewing position. 

10 The apparatus includes a CCD-camera 2 as an image-recording device, with 
an objective lens 2a which has a field of view 3, a support surface 4, prisms 
5 for beam deflection purposes and lenses 6 for beam formation in respect 
of the deflected radiation, as well as an image evaluation and storage unit 
7. A lighting device is not shown; the apparatus by way of example 

15 accordingly operates with diffuse ambient light. 

The camera 2 records an overall image of the object 1, which is 
composed of a plurality of partial images of various views of the object. The 
light which is reflected from the top side la of the object and which passes 
directly into the objective lens 2a produces a partial image of the plan view, 

20 and the light which is reflected from the side surfaces lb and which is 
deflected by the prisms 5 into the objective lens of the camera produces 
partial images corresponding to the side views. The lenses 6 produce a 
change in the imaging scale or the detailing of the recorded image in the 
parts of the beam path which are influenced by the lenses, that is to say in 

25 the partial images of the side views. 

The camera 2 transmits the overall image in the form of an electrical 
signal pattern to the image evaluation and storage device 7 (not described 
in greater detail herein) which for example can be embodied by a personal 
computer. The image evaluation and storage unit 7 determines the 

30 boundaries and demarcation of the partial images and, using image 
processing and image evaluation methods which are known per se but 
which are specifically adapted to the features to be checked in the partial 
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images, extracts relevant items of information and parameters from the 
partial images, and prepares them for comparative evaluation. 

Figure 2 shows a typical overall image 11 of a circuit in the field of 
view 10 of the camera, the image having been recorded by the camera 2 in 
the apparatus shown in Figure 1 and stored in the image evaluation and 
storage unit 7. The overall image 10 comprises five partial images, more 
specifically the partial image 11a of the plan view and the partial images 
lib through lie of the side views. Printing 12 on the top side of the circuit 
stands out clearly in its intensity from the intensity of the ambient light. 
The images of the side views are formatively influenced by superposition 
structures 13 which include elements which are caused by gaps of different 
widths between the legs of the circuit 1 and the support 4. The 
superposition structures are evaluated by means of special image 
processing methods as already referred to hereinbefore and used for a 
checking procedure in respect of coplanarity of the circuit connections. 

Figure 3 shows a detail of a modified embodiment of the 
arrangement shown in Figure 1, in which, in addition to the components 
illustrated therein, there are provided two lighting devices 8a, 8b for 
producing top light on the surface of the object 1 and for producing 
transmission light for the side views of the legs 1.1 of the circuit 1, the 
transmission light being rendered diffuse by a light diffuser device 9. 

Figure 4 shows a spline curve S for the variation in intensity in the 
pixels of the digitised partial image of a side view for one of the gaps 
between a leg and the support. The area A s under the spline curve S, 
defined by the rise points X 0 and X lf is used in the context of a specific 
image evaluation procedure for the side view partial image for computation 
of the gap width as a coplanarity parameter. 

Figure 5 denotes the parameters checked in the context of a so- 
called lead and mark inspection, by reference to the side view lb of an 
object 1 and by reference to the underneath view lc thereof. The object 
shown at the left in Figure 5 has L-shaped connections while the object 
shown at the right in Figure 5 has J-shaped connections. The parameters to 
be checked are the coplanarity CP, the housing bottom clearance ST, the 
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pitch PI, the twist or skew SK, the connection or lead width LW, the length 
LS, the connection or lead position LP, the length deviation LED, the 
connection dimensions TD, the contact points or the footprint FP and the 
housing height (not identified). 



9 



AMENDED PAGE 



Berlin 



17th July 2000 
BB9008 MK/as 



Our ref : 



Applicants/proprietors: Bi-Ber Bilderkennungssysteme GmbH Berlin 



1. A method for automated detection and checking of geometrical 
and/or textural features of an object (1) in various views, in particular in 
side views (lb) and a plan view (la), using an opto-electronic image- 
recording device (2) as well as a storage and evaluation unit (7) for image 
processing and image evaluation, wherein quality or state assessment of 
the object is effected by a comparison with parameters which are 
predetermined in respect of the individual features, wherein a plurality of 
partial images (11a through lie) of the object are substantially 
simultaneously recorded by means of a number of image-recording devices 
and beam-deflection means (5), which number is smaller than the plurality 
of partial images, and at least partially optically assembled at the same 
time to form an overall image (11) which shows all views and in which the 
boundaries of the partial images can be recognised, and the overall image 
is evaluated separately for checking the individual features in the 
boundaries of the partial images, characterised in that 

- in the regions (lib through lie) of the overall image, which show 
side views (lb) on to the object (1), locations at which the object comes 
very close to a support surface (4) are ascertained by analysis of the gray 
value distributions, 

- subsequently the light quantity which passes through between the 
object and the support surface and which is reflected in the pixels as an 
intensity value is detected, 

- the local light quantity pattern characterising the width of a gap 
between the object and the support surface is determined using the 
intensity values, and 
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New claims 
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- the light quantity pattern is converted in accordance with a 
predetermined algorithm using calibration information into a gap width 
which is present between the object and the support surface. 

2. A method as set forth in claim 1 characterised in that all partial 
images (11a through lie) are assembled optically and recorded by 
precisely one image-recording device (2). 

3. A method as set forth in claim 1 or claim 2 characterised in that in 
the overall image (11) the regions of the partial images are so positioned 
and identified, using the storage and evaluation unit (7), that they can be 
associated with the individual views (la, lb). 

4. A method as set forth in one of the preceding claims characterised 
in that in at least one additional step the scene is recorded without object 
(1) and/or with a reference object which has predetermined parameters in 
respect of the features and the corresponding overall image (11) is put in 
the storage and evaluation unit (7) for comparison and calibration 
purposes. 

5. A method as set forth in one of the preceding claims characterised 
in that in a region (11a) of the overall image which shows in particular a 
plan view (la), by means of image processing, using convolution filters, 
areas with severe local intensity differences are emphasised, detected and 
quantified in respect of their dimensions. 

6. A method as set forth in one of the preceding claims characterised 
in that integrated into the overall image (11) is a representation of the side 
of the object (1) which is remote from the or all image-recording device or 
devices (2) and which is in particular towards the support surface (4). 

7. An apparatus for automated detection and checking of geometrical 
and/or textural features of an object (1) in various views (la, lb), in 
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particular in side views and a plan view, comprising an opto-electronic 
image-recording device (2) and a storage and evaluation unit (7) for image 
processing and image evaluation, wherein there are provided optical means 
(5) for beam deflection, by means of which a plurality of partial images 
(11a through lie) of the object are substantially simultaneously recorded 
by a number of image-recording devices, which number is smaller than the 
plurality of partial images, and are at least partially assembled optically at 
the same time to form an overall image (11) which shows all views and in 
which the boundaries of the partial images are recognisable, characterised 
in that there is provided a flat support surface (4) for the object and the 
beam-deflection means (5) are arranged substantially in the plane of the 
support surface in such a way that there is a view parallel to the support 
surface, which permits checking of the coplanarity of a plurality of parts 
(1.1) of the object, which are towards the support surface. 

8. An apparatus as set forth in claim 7 characterised in that there is 
provided a single image-recording device (2), relative to which the object 
(1) is positioned in such a way that it fills only a partial region of its field of 
view which is determined by the viewing angle (3), and that disposed in 
remaining parts of the field of view are beam-deflection devices (5) which 
project side views (lb) of the object on to the image-recording device. 

9. An apparatus as set forth in claim 7 or claim 8 characterised in 
that the beam-deflection means have prisms (5) or mirrors which in 
particular are displaceable and/or have curved surfaces. 

10. An apparatus as set forth in claim 7 or claim 8 characterised in 
that the beam-deflection beams have a light guide device. 

11. An apparatus as set forth in one of claims 7 through 10 
characterised in that associated with at least one of the beam-deflection 
means are means (6) for changing the imaging scale of at least one partial 
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image with respect to at least one other partial image, in particular a lens 
arrangement. 

12. An apparatus as set forth in one of claims 7 through 11 
characterised by a lighting device (8a, 8b, 9) which in particular has a light 
diffuser device (9) for producing a uniform light flux under the object (1), 
which is arranged behind projecting parts (1.1) of the object. 

13. An apparatus as set forth in claim 12 characterised in that the 
light diffuser device (9) is interrupted in such a way as to permit a view on 
to the side of the object (1), which is remote from the or all image- 
recording device or devices (2). 

14. An apparatus as set forth in one of claims 7 through 13 
characterised in that the image-recording device (2) and the storage and 
evaluation unit (7) are integrated to form a structural unit. 
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Abstract 



A method for automated detection and checking of geometrical 
and/or textural features of an object (1) in various views (la, lb), in 
particular in side views and a plan view, using an opto-electronic image- 
recording device (2) as well as a storage and evaluation unit (7) for image 
processing and image evaluation, wherein quality or state assessment of 
the object is effected by a comparison with parameters which are 
predetermined in respect of the individual features and wherein a plurality 
of partial images of the object are substantially simultaneously recorded by 
means of a number of image-recording devices and beam-deflection 
means, which number is smaller than the plurality of partial images, and at 
least partially optically assembled at the same time to form an overall 
image which shows all views and in which the boundaries of the partial 
images can be recognised, and the overall image is evaluated separately 
for checking the individual features in the boundaries of the partial images. 
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